Remote vehicles, such as unmanned ground, air, or surface vehicles, are typically controlled by a human operator using a console or portable computer device. The operator is required to manually actuate numerous buttons and control inputs (e.g., joysticks, mouse, pucks, levers, etc.) corresponding to the controls of motion axes and subsystems of the remote vehicle. Conventional remote vehicle control requires a trained and skilled operator, and demands concentration on the control device to efficiently drive the remote vehicle to a destination.
In addition, remote vehicles are increasingly being used in military, law enforcement, and industrial applications to provide a tool for a person to perform operations at a safe, remote distance from sites of potential danger or hazard to human beings. Such remote vehicles are being deployed for some tasks by military and civilian forces, such as bomb and ordnance disposal, in which the remote vehicle is remotely navigated to the proximity of the explosives or other potentially dangerous target by an operator located hundred of meters away, so that investigation and disarmament can take place at a safe distance.
FIG. 1 illustrates a conventional remote vehicle being wirelessly controllable at distances of hundreds of meters or more from the operator, depending presently on the limits of available wireless communication.
In typical remote vehicle operation, the operator controls the vehicle using a process known as tele-operation. Conventional remote vehicle tele-operation involves the use of operator control consoles, most commonly having joysticks, trackballs, mouse-type input devices, or some arrangement of physical switches and/or potentiometers and similar manual actuation input devices. Remote vehicles are typically configured with many axes of motion, including motion drive axes, steering axes (either physical or derived virtual steering), manipulation axes, sensor pan-tilt-zoom axes, etc. The axes of the remote vehicle often involve complex mechanical coupling between the drive actuators and the physical motion apparatus, such as wheels, tracks, rudders, heads, etc. Additionally, remote vehicle platforms typically contain many sensors, such as cameras, that can provide multiple streams of video to the operator as visual feedback to aid the operator's control. The electro-mechanical complexity of many remote vehicles has consequently made the manual control of such vehicles complex for human operators in a tele-operation process, requiring many function-specific knobs, joysticks and buttons to perform a task (see, e.g., the conventional remote vehicle control console illustrated in FIG. 2). A significant amount of operator training ad experience can be required to develop sufficient manual dexterity and skill to be able to accurately navigate and control a remote vehicle. Operation of the remote vehicle require the operator's attention, diverting it from situational awareness.
FIG. 2 illustrates conventional remote vehicle control console, which can be used to control a mobile robot in military operations.
In various military applications, such as bomb disposal, it is known to have a bomb disposal specialist that is sufficiently trained and expert in operating the remote vehicle. To accomplish a bomb inspection or disposal task, the specialist uses many manual control knobs, levers, and joysticks, normally requiring two hands to operate and close attention to a video console. Other personnel must provide security for the remote vehicle operator and perform other tasks, because the remote vehicle operator is typically fully engaged with the complex task of tele-operation. For many tasks, such complex and attention-demanding control devices are prohibitive to the safe and efficient accomplishment of the tasks. For example, the use of remote vehicles in forward reconnaissance and patrol missions by dismounted infantry soldiers can be an operational and tactical impracticality due to the cumbersome and immersive nature of existing remote vehicle tele-operation controls.
In order for robots to be beneficial in such military activities, a method and device for directing the actions of the remote vehicle is needed that is more integrated to the normal mission actions and more intuitive to use. The device must not overly encumber the soldier/operator to the point of endangering other key operational goals, such as constant vigilance to security and ready access to defensive weapons or other tools.
For example, conventional remote vehicle controllers typically require two-handed operation or otherwise highly constrain the motion and the visual attention of the operator. Control devices such as game controllers, commonly used for video gaming systems, offer a familiar format and reasonably good ergonomics, but normally require the use of both hands by the operator. More complex robot control schemes have also been implemented using map-based graphical user interfaces, on which an operator may designate a path and a destination corresponding to Global Positioning System (GPS) coordinates. While these methods simplify the task of controlling a remote vehicle, such approaches require high resolution computer graphical displays, and complex information about the terrain and obstacles. Such GPS coordinate systems can be subject to failure or limited usefulness due to unavailable or inaccurate GPS satellite data, including indoor and underground environments where GPS satellite signals cannot be received.
To extend the usefulness of remote vehicles, a control system and methodology is needed that can be more effective in many environments and with minimal intrusion on the human operator's freedom of motion, focus, and action.
A weapon-mounted hand-held controller having a gyroscopic motion tracker is known for use with ‘virtual reality’ training and simulation of combat, in which the hand-held controller is used to move the soldier's actor view through simulated realities that are displayed graphically through a full immersion head mount display. When the weapon-mounted single-hand controller is used in such a ‘virtual reality’ mode, the operator may change his own apparent position in the graphically-generated world image that is displayed on his head-mount viewer. A motion tracker incorporated in the hand-held controller is used to track the aiming of the soldier's weapon in the virtual world and to determine the apparent target of simulated weapon firings. This system has been proven effective and not overly encumbering to the actions of a soldier in training simulations of a wide range of dismount combat situations (see Quantum3D Expedition DI product, for example).